1 Proactive Fault - Recovery in Distributed Systems

Supporting both real-time and fault-tolerance properties in systems is challenging because real-time systems require predictable end-to-end schedules and bounded temporal behavior in order to meet task deadlines. However, system failures, which are typically unanticipated events, can disrupt the predefined real-time schedule and result in missed task deadlines. Such disruptions to the real-time schedule are aggravated in asynchronous distributed systems by two main factors: first, delays in failure detection, and second , increased latencies due to the reactive fault-recovery routines that are set into motion once a failure is detected. In this thesis, we present a general framework for proactive (rather than the classical reactive) fault-recovery that reduces the latencies incurred by the fault-recovery routines. Proactive fault-recovery is a technique that exploits fault prediction mechanisms in order to compensate for failures even before they occur, thereby providing bounded temporal behavior in real-time and fault-tolerant systems for certain classes of faults. In our framework, we also show how to exploit knowledge of the underlying system topology to apply the benefits of proactive fault-recovery to multi-tiered distributed systems. We evaluate the impact of the design choices we faced when implementing a prototype of this framework in a distributed CORBA application. Our preliminary results show a promising 76% reduction in the worst-case fault-recovery latencies in our application. This demonstrates that proactive fault-recovery can indeed provide bounded temporal behavior in the presence of certain kinds of faults, thereby facilitating the development of real-time, fault-tolerant distributed systems.